Papers with knowledge representation
CogKTR: A Knowledge-Enhanced Text Representation Toolkit for Natural Language Understanding (2022.emnlp-demos)
Copied to clipboard
Zhuoran Jin, Tianyi Men, Hongbang Yuan, Yuyang Zhou, Pengfei Cao, Yubo Chen, Zhipeng Xue, Kang Liu, Jun Zhao
| Challenge: | Existing knowledge-enhanced methods are limited to knowledge-intensive tasks. |
| Approach: | They propose a knowledge-enhanced text representation toolkit for natural language understanding . it combines knowledge acquisition, knowledge representation, knowledge injection and knowledge application . |
| Outcome: | The proposed toolkit supports knowledge acquisition, knowledge representation, knowledge injection, and knowledge application. |
Knowledge-Augmented Methods for Natural Language Processing (2022.acl-tutorials)
Copied to clipboard
| Challenge: | Knowledge in natural language processing (NLP) is a rising trend especially after the advent of large scale pre-trained models. |
| Approach: | This tutorial introduces the key steps in integrating knowledge into natural language processing (NLP) it introduces knowledge grounding from text, knowledge representation and fusing. |
| Outcome: | This tutorial introduces the key steps in integrating knowledge into natural language processing including knowledge grounding from text, knowledge representation and fusing. |
Formal Semantic Controls over Language Models (2024.lrec-tutorials)
Copied to clipboard
| Challenge: | Text embeddings provide a concise representation of the semantics of sentences and larger spans of text, rather than individual words, capturing a wide range of linguistic features. |
| Approach: | They propose to shorten the gap between latent semantics and formal symbolics by comparing distributional models to symbolic models grounded on formal linguistics and well-defined mathematical properties. |
| Outcome: | This paper examines the analysis and control of text representations, covering methods from pooling to LLM-based. |
Pingan Smart Health and SJTU at COIN - Shared Task: utilizing Pre-trained Language Models and Common-sense Knowledge in Machine Reading Tasks (D19-60)
Copied to clipboard
| Challenge: | Existing approaches to represent knowledge in the low-dimensional space are to leverage large-scale unsupervised text corpus to train fixed or contextual representations. |
| Approach: | They propose to leverage large-scale unsupervised text corpus to train fixed or contextual language representations and to express knowledge into a knowledge graph (KG) they incorporate distributional representations of a KG onto the representations from pre-trained language models, via simply concatenation or multi-head attention. |
| Outcome: | The proposed models outperform the other models on the COIN: COmmonsense INference in Natural Language Processing (COIN) Workshop datasets. |
A Survey on Automated Fact-Checking (2022.tacl-1)
Copied to clipboard
| Challenge: | Fact-checking is an essential task in journalism due to the speed with which information and misinformation can spread in the media ecosystem. |
| Approach: | They propose to use natural language processing to automate fact-checking by identifying common concepts and defining definitions. |
| Outcome: | The proposed method can predict the veracity of claims using natural language processing, machine learning, and databases. |
KGLM: Integrating Knowledge Graph Structure in Language Models for Link Prediction (2023.starsem-1)
Copied to clipboard
| Challenge: | Knowledge graphs are incomplete in the information they represent, necessitating knowledge graph completion tasks. |
| Approach: | They propose a new entity/relation embedding layer that learns to differentiate distinctive entity and relation types, thus allowing the model to learn the structure of the knowledge graph. |
| Outcome: | The proposed language model learns to differentiate distinct entity and relation types, thus learning the structure of the knowledge graph. |
Pretrain-KGE: Learning Knowledge Representation from Pretrained Language Models (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding models suffer from limited knowledge representation due to sparse and noisy dataset annotations. |
| Approach: | They propose to use pretrained language models to enhance knowledge representation by leveraging world knowledge from pretrained models. |
| Outcome: | Extensive experiments show that the proposed framework can improve results over existing models. |
DomiKnowS: A Library for Integration of Symbolic Domain Knowledge in Deep Learning (2021.emnlp-demo)
Copied to clipboard
| Challenge: | Current deep learning architectures are data-hungry with issues mainly in generalizability and explainability. |
| Approach: | They propose a library for the integration of domain knowledge in deep learning architectures . structure of data is expressed symbolically via graph declarations and constraints can be added to deep models . |
| Outcome: | The proposed framework simplifies programming for integration of domain knowledge in deep learning architectures while separating the knowledge representation from learning algorithms. |
Efficient Federated Learning on Knowledge Graphs via Privacy-preserving Relation Embedding Aggregation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing frameworks that share entity embeddings of knowledge graphs (KGs) would incur a severe privacy leakage. |
| Approach: | They propose a new attack method that aims to recover the original embedding information based on the known entity embeddables of FedE. |
| Outcome: | The proposed framework can be used to infer whether a specific relation exists in a private client. |
Modeling Semantic Plausibility by Injecting World Knowledge (N18-2)
Copied to clipboard
| Challenge: | Existing models for semantic plausibility are based on distributional data, but injecting knowledge about entity properties provides a substantial performance boost. |
| Approach: | They propose to inject manually elicited knowledge about entity properties into a dataset to improve plausibility models. |
| Outcome: | The proposed dataset is a great testbed for semantic plausibility models . it shows that injection of knowledge about entity properties improves performance . |
Bridging the Embodiment Gap in Agricultural Knowledge Representation for Language Models (2025.acl-srw)
Copied to clipboard
| Challenge: | a paper quantifies the “embodiment gap” between disembodied language models and embodied agricultural knowledge communication . agronomists and researchers examined the embodiment gap in 78 farmers . |
| Approach: | They propose a framework that integrates linguistic patterns from five domains of agricultural expertise and a new metric for evaluating embodied knowledge representation in language models. |
| Outcome: | The proposed frameworks reduce the embodiment gap by 47.3% across agricultural domains . the proposed framework improves tool usage discourse and soil assessment terminology . |
Relational World Knowledge Representation in Contextual Language Models: A Review (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge bases are organized according to manual schemas that limit their expressiveness and require significant human engineering and maintenance. |
| Approach: | They propose to organize knowledge representation strategies in LMs by the level of KB supervision provided . they propose to highlight notable models, evaluation tasks, and findings . |
| Outcome: | The proposed model can internalize and express relational knowledge in more flexible forms. |
What Action Causes This? Towards Naive Physical Action-Effect Prediction (P18-1)
Copied to clipboard
| Challenge: | a new task on naive physical action-effect prediction addresses the relationship between concrete actions and their effects on the state of the physical world as depicted by images. |
| Approach: | They propose a task that harnesses web image data to facilitate action-effect prediction. |
| Outcome: | The proposed approach harnesses web image data through distant supervision to facilitate learning for action-effect prediction. |
PREE: Towards Harmless and Adaptive Fingerprint Editing in Large Language Models via Knowledge Prefix Enhancement (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing black-box fingerprinting techniques rely on overfitting high-perplexity trigger patterns . experimental results show that model editing in the fingerprint domain exhibits unique advantages . |
| Approach: | They propose a prefix-enhanced fingerprint editing framework that encodes copyright information into parameter offsets through dual-channel knowledge edit to achieve covert embedding of fingerprint features. |
| Outcome: | The proposed model editing framework achieves 90% trigger precision in mainstream architectures . the proposed model editor achieves the 90% accuracy in mainstream models . |
AdaEdit: Advancing Continuous Knowledge Editing For Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge editing methods that can efficiently update knowledge in LLMs are limited due to budget constraints. |
| Approach: | They propose a method that can enhance the performance of edited LLMs in large-size continuous editing regimes. |
| Outcome: | Extensive empirical evaluations on multiple LLMs show that the proposed method outperforms existing methods without compromising the general abilities of these models. |
Differentiating Concepts and Instances for Knowledge Graph Embedding (D18-1)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding methods encode concepts and instances as vectors in a low-dimensional space, ignoring the difference between concepts and instance. |
| Approach: | They propose a knowledge graph embedding model that separates concepts from instances by differentiating concepts and instances. |
| Outcome: | The proposed model outperforms state-of-the-art methods on link prediction and triple classification tasks on YAGO dataset. |
Efficient and Effective Prompt Tuning via Prompt Decomposition and Compressed Outer Product (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning pre-trained language models overlook intrinsic semantic associations between soft prompt tokens, leading to high discreteness and limited interactions. |
| Approach: | They propose a low-parameters Prompt Tuning method which leverages prompt decomposition and compressed outer product to facilitate multiple interactions among prompt tokens. |
| Outcome: | Experiments on six architectures and eight datasets show that the proposed method outperforms state-of-the-art methods in performance and efficiency. |
Automated Fact Checking: Task Formulations, Methods and Future Directions (C18-1)
Copied to clipboard
| Challenge: | Recent research on fact checking has focused on misinformation . however, relevant papers and articles have been published in research communities that are unaware of each other and use inconsistent terminology. |
| Approach: | They propose avenues for future NLP research on automated fact checking . they highlight the use of evidence as an important distinguishing factor . |
| Outcome: | The proposed methods unify the task formulations and methodologies across papers and authors. |
Role-Sensitive Neurons: A Neuron-Level Gain Control Mechanism for Confidence Steering (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit striking behavioral flexibility. |
| Approach: | They propose to identify a sparse sub-network of Role-Sensitive Neurons (RSNs) that governs the transition from hesitation to action. |
| Outcome: | The proposed framework allows precise regulation of abstention behavior by intervention on this subspace. |
TEMP: Taxonomy Expansion with Dynamic Margin Loss through Taxonomy-Paths (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing taxonomies are unable to maintain coverage due to the rising of new concepts . TEMP uses pre-trained contextual encoders to predict the position of new ideas . |
| Approach: | They propose a self-supervised taxonomy expansion method that ranks taxonomies by ranking them . they use pre-trained contextual encoders to train the model with dynamic margin loss . |
| Outcome: | The proposed method outperforms state-of-the-art taxonomy expansion methods by 14.3% and 15.8% on public benchmarks. |
PropRAG: Guiding Retrieval with Beam Search over Proposition Paths (2025.emnlp-main)
Copied to clipboard
| Challenge: | Retrieval Augmented Generation (RAG) is a non-parametric approach for large language models. |
| Approach: | They propose a framework that shifts from triples to context-rich propositions and introduces an efficient, LLM-free online beam search over proposition paths to discover multi-step reasoning chains. |
| Outcome: | The proposed framework achieves state-of-the-art zero-shot Recall@5 and F1 scores on 2Wiki, HotpotQA, and MuSiQue. |
COPEN: Probing Conceptual Knowledge in Pre-trained Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge probing studies focus on evaluating factual knowledge of pre-trained language models (PLMs) but ignore conceptual knowledge. |
| Approach: | They evaluate conceptual knowledge of pre-trained language models by annotating 24k data instances covering 393 concepts. |
| Outcome: | The proposed tasks evaluate pre-trained language models' conceptual knowledge of entities, learn conceptual properties, and conceptualize entities in contexts. |
Leveraging 3D Gaussian for Temporal Knowledge Graph Embedding (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Representation learning in knowledge graphs (KGs) has focused on static data, yet many real-world knowledge graph are inherently dynamic. |
| Approach: | They propose a temporal embedding method inspired by 3D Gaussian Splatting where entities, relations, and timestamps are modeled as 3D gaussian distributions with learnable structured covariance. |
| Outcome: | The proposed method outperforms state-of-the-art methods on three benchmark TKG datasets. |
ECoK: Emotional Commonsense Knowledge Graph for Mining Emotional Gold (2024.findings-acl)
Copied to clipboard
Zhunheng Wang, Xiaoyi Liu, Mengting Hu, Rui Ying, Ming Jiang, Jianfeng Wu, Yalan Xie, Hang Gao, Renhong Cheng
| Challenge: | Existing knowledge graphs focus on the representation and reasoning of general factual knowledge, while there are significant deficiencies in the understanding and reasoning for emotional knowledge. |
| Approach: | They propose a commonsense knowledge graph that can be used to represent emotional knowledge by combining theories from psychology, cognitive science, and linguistics. |
| Outcome: | The proposed model surpasses GPT-4-Turbo in the emotion-related tasks. |
UniLR: Unleashing the Power of LLMs on Multiple Legal Tasks with a Unified Legal Retriever (2025.acl-long)
Copied to clipboard
Ang Li, Yiquan Wu, Yifei Liu, Ming Cai, Lizhi Qing, Shihang Wang, Yangyang Kang, Chengyuan Liu, Fei Wu, Kun Kuang
| Challenge: | Existing retrieval methods are designed for general domains, struggling with legal knowledge, or tailored for specific legal tasks, unable to handle diverse legal knowledge types. |
| Approach: | They propose a novel retrieval method that integrates specialized knowledge into LLMs. |
| Outcome: | The proposed method can perform multiple legal retrieval tasks for LLMs. |
Time Course MechInterp: Analyzing the Evolution of Components and Knowledge in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models acquire and store factual knowledge for interpretability, reliability, efficiency . prior work on factual recall focused on localizing knowledge within transformer parameters . |
| Approach: | They analyze the evolution of factual knowledge representation in a large language model by tracking its attention heads and feed forward networks over training. |
| Outcome: | The proposed model acquires and stores factual knowledge over time and is adaptively trained . the proposed model can be pruned, optimized, and transparent . |
Dynamic Heterogeneous-Graph Reasoning with Language Models and Knowledge Representation Learning for Commonsense Question Answering (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for QA use knowledge graphs, but they ignore subgraph optimization and subgraph deepening. |
| Approach: | They propose a dynamic heterogeneous-graph reasoning method with LMs and knowledge representation learning that optimizes the structure and knowledge representing of the HKG using a two-stage pruning strategy and knowledge-representation learning. |
| Outcome: | The proposed method improves on existing methods at CommonsenseQA and OpenBookQA. |
Fisher-Driven Adaptive Locating for Knowledge Editing in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for locating and editing static knowledge are costly and risk catastrophic forgetting or error. |
| Approach: | They propose a Fisher-driven adaptation-aware locating strategy that dynamically identifies which model components should be edited for a given knowledge update. |
| Outcome: | Experiments on standard benchmarks show that FiDAL improves editing effectiveness and knowledge preservation across multiple editing methods. |
Synthetic Knowledge Ingestion: Towards Knowledge Refinement and Injection for Enhancing Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models capture factual knowledge across a wide range of domains, but refining their capabilities on previously seen knowledge remains a challenge. |
| Approach: | They propose a synthetic knowledge ingestion method that leverages fine-grained synthesis and interleaved generation to construct high-quality data representations from raw knowledge sources. |
| Outcome: | The proposed method outperforms baseline methods on question-answering tasks spanning finance, biomedicine, and open-generation domains. |
SocraticKG: Knowledge Graph Construction via QA-Driven Fact Extraction (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to construct knowledge graphs struggle with factual coverage and information loss. |
| Approach: | They propose an automated KG construction method that introduces question-answer pairs as a structured intermediate representation to unfold document-level semantics prior to triple extraction. |
| Outcome: | The proposed method achieves superior factual retention while maintaining high structural cohesion even as extracted knowledge volume substantially expands. |